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ABSTRACT 

Fiindaaental problems in the collection and use of 
data# in particular the need to develop and uso criteria for data, 
are discussed. Specific examples of the use and misuse of data 
related to teachers and children in the public schools are presented- 
Two methods of data collection-*manitest techniques and formal 
response instruments — are evaluated in teras of advantages and 
disadvantages# as well as reliability and validity. Additional 
consideration is given to criteria for pre-colLection data phases# 
and test selection {ability and achievement tests). (AE) 
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take our text from a 1967 Report of the United States Commission 
on Civil Rights, In which the following testlmonv Is given by sn 
Aosociate Superintendent of Schools of one of our largest cities: 



• •there v*as a teacher who had copied the IQ 
numbers dorm the line from s list In the 
principal's of flee. • .Throughout the semester 
If the teacher ca}lc<l on Mary, let us say, with 
an IQ of 119, she followed somewhat this patCrern: 
If Mary didn't respond quickly, ^Vell, now, 
come on, Kary. You know you can do this. You 
know how we did this yesterday,” or brltig up 
an analogoui situation. She encouraged, she 
-.tluulated, until Maty came up ulth the proper 
answer, or what the teacher at Jcist consldotcd 
an adequate one. However, when she called on 
poor John with his 7^ IQ, If he mumbled 
something fairly audible, why, this was 
wonderful; pat him on the bock and, ”Be 
sure aiwJ be here tomorrow. You con vnsh the 
windows and help move the plar.o and water the 
flowers, and the erasers* must bo woohed,'^ and 
so forth. This Is the k^nd of encouragement 
that he got with s 74. This Is tcachlrtg by IQ. 
She was a little horrified at the ^nd of the 
semester when she turned In her grsdae. She 
looked under the glass and saw that the columns 
she had coplel for IQ's were locker numbers. 

Sow, this is about what goes on. 
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There are many reasons why one may wish to collect data about 
ar. educational system, or any system for that matter. The > ~.use may 
be a dissertation, a need to .lend support to a political decision, a 
need to defend and preserve an Institution, and so forth. 

Also, one may wish to collect data for research, experimentally 
designed to discover new knowledge about curricular or methodological 
alternatives. Or one may simply want to describe the system, and 
descriptive data are gathered. 

A neglected but important reason to gather data, particularly for 
national and regional educational planners, is to develop viable 
alternatives for decisionmakers. In the term 'Meclslon-makers^^ 
we Include you and those above you In the substantive as well as 
resource-allocation lines of authority. 

The question la what data. lii our vlew„ the collectiors of dati 
are economic activities In the sense that we are dealing with act^rce 
resources — time, energy, material, machlrery and good will — neocoe.ity 
to make and carry out decisions about what data, when. In what quantity 
and from whom they will be gathered ; In order to collect, process, 
analyze, and report the data and In order to obtain good raw material 
from the source. 

It Is therefore important that we develop and use criteria for 
dataj not only because of th** scarce resources which necessarily go 
Into their collection and use, but also because they have to serve 
ovjr puepooo nud needs onre, rnllectcd. 

Let us consider some specific examples with regard to children 
and teachers In the public schools. 

The system typically collects a good deal of data on children In 
school, but we suspect that the most widely available and quoted data 
are derived from standardized tests, such as achievement tests and 
so called "ability tests." one often finds It difficult to publicly 
explain or defend the use to which such test results are put. 

Research so far has not supported their use as devices for placing 
students on tracks or In streams. They are not widely used for diagnostic 
purposes nor In other ways that would help the child do better In 
school. They are seldom used for any respectable educational research 
except In the case of specially designed studies like the Coleman 
study In Equal Educational Opportunity (1966, Washington, D.C., U.S. 
Government Printing Office). 

Of course, there Is variance among tests, and some exceptions 
to the above can always be cited, but In general these criticisms are 
not unfair. It Is important to note, however, that the criticisms 
are directed not at the tests but at the use, lack of use, or misuse 
of the teats. Thus, If standardized achievement tests have some 
predictive capacity with regard to later school achievement, what 
positive action do we base on such data? In the negative sense, of 
course, one can always cite examples of the use of test data to create 
or reinforce Impressions of ability or dullness which may not be 
valid. 
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A thought bordering on heresy: what calamities would fisult In 
schools If ve did away wivh standardized achievement and ability 
tests? 

Now let'fe^ go to the subject of teachers. The teaching staff is 
u aanlpuablei variable; unlll'e students, home and family environment, 
deep-seated attitudes, and heredity, which seem to us rather important 
but rather fixed variables in student success at learning, the teaching 
staff (as well as other variables like facilities, curriculum, methods) 
is not fixed. It Is always in the process of chanf,Jng and, we hope, 
growing. Yet we collect little useful data on teachers, or at least 
we collect the wrong data, and in any case don't Intelligently use 
the data we collect. 

For example, Coleman data show that teacher verbal skill, not 
level of education nor teacher experience, la most strongly related 
to achievement of children in school, at least In regard to the 
traditional Indices of school quality. Yet our data on teacher 
experience and education are ouch more carefully collected and more 
widely used In determining who gets id, placed and rewarded. 

It may be that If we were more rational, ve would begl^, to collect 
data on the charaeterlstics of successful teachers, then try to 
r?pMcate tliese characteristics in the selection and training of 
teachers. 

We may find that some teachers are right for some children and 
wrong for others. One study* suggests that "since different students 
will respond differently to different styles, attitudes, activities, 
language, strictness, etc. these properties of teacher activities 
should be investigated, ' Clearly, If "teacher specificity" is 
important to pupil achievement, the data collected on teachers will 
have to be greatly changed and augmented. 

To avoid belaboring a point, it Is time to tie some thoughts 
together. First, we have to be hones< with ourselves In selecting 
data to be obtained. If our educational objective Is the preparation 
of good men and women --however ve define good — we honestly can't 
tell how well we do that by gathering achievement test data. That 
must be a very small aspect of the life of the good men. 

Second, ve have to carefully analyze and dissect our goals 
so rhst we can develop criteria for research and related data 
collection. It doesn't make much sense to gather data on the 
height of each child If the goal of the school Is unrelated to 
height. 

Third, having uioie decisions about, for example, curricular 
methodological and teaching sttategied, It will be necessary to gather 
only the specific data that will Indicate progress or lack of progress. 



* Stephen Mlchelson, Association of Teacher Resources With 

Chlldren'a Characteristics" In Do Teachers Make a Difference, U.S, 
Government Printing Office, Washington, D,C*, 1970 
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Tha more perfortnance-crlented the data Is, the better; but the 
Inability to gather performance data, or Indeed any relevant data at 
all, does not Ipso facto mean that the strateg)^ or goal Is poor* 

It means rather that ve do not have the tools to measure It; and that 
lidmlsslon might be made much more often by those whose business Is the 
collection of da):a« 

Nov, having discussed the use and collection of data at a high 
level of abstraction, let us admit that we have to face data problems 
today and deal with them as Intelligently and ^sensitively as we can. 
For that reason we have put tdffether a neoftssprlly brief dfS'cu^lon 
for you which Is more practically oriented. 

A great many data are routinely available to the school 
administrator. Attendance records^ grade averages. Intelligence 
scores, drop-out rates, to mention a few examples, are collected on 
a routine basis by almost all school systems. Often little thought 
Is given to what use Is to be made of this Information and, as a 
result, it Is usually collected, stored and forgotten. On the rare 
occasion when these data are examined for possible Information, It Is 
usually discovered that either the wrong data have' been '^ll^dcedcerr 
are Incomplete. It Is of prime Importance that due consideration be 
given to what questions the data are expected to answer before any 
collection Is Initiated. 

Once It Is decided that the collection of data Is necessary to 
help solve seme problem, a decision has to be made as to how the data 
are to be collected. For the purposes of this paper we will discuss 
two methods for collecting Information: response Instruments and 

manifest techniques. 

Manifest Techniques. No matter what name Is given to a formal 
response technique, and regardless of whether It Is measuring 
academic achievement or attitude towards school, all such paper and 
pencils tests are basic/ lly cognitive. As an alternative to the 
collection of Information by Che use of formal response Instruments, 
consideration should be given to the use of manifest techniques. 

What ve mean by a manifest technique Is best shown by an example. 

In a school project It Is decided that It la Important to measure 
the degree of interest that parents show towards the operation cf the 
school. As a result of this decision, an attitude scale Is drawn 
up and administered to a random sample of parents. Thus we have a 
formal response Instrument being used to gather data. It Is then 
reasoned that If any change does take place In the attitude of parents 
towards the school this change will be reflected In their attendance 
at PTA meeting. A count Is then made at each scheduled PTA meeting 
and a record kept. ?.n this situation parents expressed their attitude 
toward the school through th'j manifestation of their Involvement In 
school activities* 

One of the principal advantages that this data collecting 
technique has over the use of formal response Instruments Is that 
It can be non-reactlve* Often when formal response Instrumenta are 
administered the change that takes place In the subject Is due as much 
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to the test instruiuertts as to the treatment. When a manifest technique 
is used the eubj^.cts are often unaware that any data are being collected 
and do not alter their behavior because of a testing situation. 

This brief mention of the use of manifest data in this paper 
has been made to alert the reader of Its existence. For a more 
in-depth treatment of the subject, the reader is referred to the 
excellent text by Webb et al. titled. Unobtrusive Measure r . !Nor.reactive 
Research in the S ocial Sciences. 

R esponse InstruciEnts . By response instruments we mean those 
situations where the respondent makes a formal reply to specific 
questions or statements. Examples of Instzuments used in this method 

are standardized tests, teacher-made tests, attitude scales, interview 
schedules and questionnaires. Of these, the standardized test is 
one of the most important instruments. In selecting a standardized 
test, consideration should be given to the validity and reliability 
of the instrument. Also, a careful investigation of the existing 
norms should be made before a final selection takes place. 

Validity is concerned with the question, ^'Does the test measure 
what it is supposed to measure?''; reliability asks, ''Does the 
test produce a consistent, accurate result?"; final] y, usability, 
which refers to the question, "Can the test be used with a minimum 
expenditure of tine, energy and money?" Beyong these three criteria, 
the following are some methods of selecting^the best test for 
evaluation purposes. 

FOXNTS ’ CO COI^SIPER BEFORE COLLECT XW DAT 

1. Why is the instrument being given? For example, why 
are standardized tests gKen in school systems? David 
A. Goslin in Teachers and Testing . 1967, summarizes the 
reasors g;iveii by public secondary school administrators 

for the use of standardized tests in 8chools(see Appendix I). 

2. Vliat wi ll be done with the results? A question such as, 

"What use will be made of the result from this test by 
students, teachers or administrators?" has to be 
answered before an appropriate instrument can be 
selected. David A. Goslln, in Teachers and T eating , 

1967, summarizes the general purposes of standardized 
tests given by public sec:>ndary school administrators. 

(nee Appendix II). 

3. Has the information already been collect ed bv some othex 
agency? If it has, why duplicaf^ ci.JeTfort? A 

little research effort •^cftool administrators will 
often save tH- s^eat deal of time, energy and money. 

is not intended to be definitive, but is given as an 
, ,/Ae of the types of questions an adolnlstrstor or teacher should 
ask himself before deciding to initiate some testing program. 
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!♦ Reputation ^ It Is simple to ask others about a test 
which they have used and vhat their feeling, s are about 
Its effectiveness* 

2* Kental Measurements Yearbook - This book may be used in 
two ways; The yearbook acquaints teachers with the 
tests available In the various fields of Instruction 
and guidance and reading the reviews of each test 
helps to reduce the number of tests to be considered. 

3. Manual of the Test - Almost all tests are accompanied 
with a manual that describes the test* how It vras 
standardized* and reports the validity and reliability of 
test. The manual also gives directions for administering 
and scoring the test* 

Obtain a Sample Set - This is the culminating step in 
the selection of a test. This permits the instructor 
to take the test himself and relate it to bis purposes 
and pupils. 

The following lists summarizing the uses of standardized tests 
and the tables in the appendix are commonly used to Illustrate the 
use of tests* We think it would be a very useful and instructive 
exercise for you, Individually or in small groups, to go through 
the lists and item by item, picking out the following; 

Which of these uses are commonly found In the schools? 

(How are tests actually used in the schools?) 

Vfhlch of these uses are educationally sound? 

Which of these uses are practicable for the teacher? 

An Ability Tes t is used ; 

1. To determine the at which the pupil Is capable of learning. 

(The mental a^;e of the child reported In years and month 
Is a good index of the level In the curriculum at which 
the child will be most capable of working.) 

2. To determine the rate at which the pupil learns. 

(The intelligence quotient Is an index of the rate of learning.) 

3. To determltie the position of a pupil in relation to others* 

4. To assist in general ability grouping* 

5. To determine the structure and pattern of ability fot 
educational and vocational guidance. 

6. To determine whether ability le a factor In a learning or 
behavior problem. 

7. to aid the teacher In setting reasonable standard of 
attainment. 
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8. To provide Inforaatlor for better Instruction In the 
areas of methods and materials • 

An Achievement Test l£ used; 

1. To determine the current achlevemert status of the 
Individual student. 

2. To evaluate the achievement of the pupil In relation 
to his actual grade placement and intellectual grade 
placement. 

3. To determine the strong and weak areas of Individual 
and group achievement. 

4. To analyze the causes of Individual and group difficulties 
and then to plan remedial action. 

5. To help the teacher evaluate his own Instruction. 
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